Summary

A convolution neural network was trained via transfer learning to identify Bactrocera species using wing images. Additional tephritids were included but only identified to genus. Threefold cross-validation was used to achieve an accuracy of 97% (845/870) at predicting class. The image set comprised 870 images in 18 classes. The image set was sorted into three folds using stratified round-robin cross-validation. Each set had a 2/3 training, 1/3 test split.

Image pre-processing

Images were prepared by:

  • Converting colour images to grayscale
  • Blurring to reduce image grain noise
  • Standardising brightness and contrast to mean = 0.5, contrast range = ±2 standard deviations, with the extreme values clipped
  • Squashing and downsampling to 299 by 299 pixels to match the network input filter size


Model training

Transfer learning was used to retrain an Xception model initially trained on the Imagenet dataset. The model was retrained three times, each run was initially set for 200 epochs but with an early-stopping callback and learning rate of 1e-04. Each model was fine-tuned at a learning rate of 1e-05, again with early-stopping set.

Images were augmented to prevent over-fitting and increase the robustness of the model to variation in additional images added later. Image augmentations included:

  • Random horizontal and vertical shift up to 10%
  • Random zoom up to ±10%
  • Random rotation up to ±25 degrees


Results

The model achieved an average accuracy of 97% (845 correct/25 wrong) across the three runs. Each model stopped before reaching 200 epochs with accuracy and loss both plateauing (Figure 1).


Fig. 1: Accuracy and loss of the three cross-validated Xception models.

Fig. 1: Accuracy and loss of the three cross-validated Xception models.


Classification report for each class

Validation accuracy varied between classes, which was to be expected given the relatively small dataset for some classes.

class precision recall f1-score support
austrotephritis 0.96 1.00 0.98 48
bactrocera_curcubitae 1.00 1.00 1.00 19
bactrocera_distincta 1.00 1.00 1.00 36
bactrocera_dorsalis 0.90 0.97 0.93 29
bactrocera_facialis 0.86 0.69 0.77 26
bactrocera_frauenfeldi 0.95 1.00 0.98 20
bactrocera_kirki 0.94 0.88 0.91 17
bactrocera_melanotus 0.96 0.92 0.94 24
bactrocera_passiflorae 0.75 0.78 0.77 23
bactrocera_psidii 1.00 0.50 0.67 4
bactrocera_tryoni 0.79 0.88 0.83 17
bactrocera_umbrosus 1.00 1.00 1.00 19
bactrocera_xanthodes 0.87 0.96 0.92 28
ceratitis 1.00 1.00 1.00 32
procecidochares 1.00 1.00 1.00 19
sphenella 1.00 1.00 1.00 10
trupanea 1.00 1.00 1.00 471
urophora 1.00 1.00 1.00 28
accuracy NA NA 0.97 870
macro avg 0.94 0.92 0.93 870
weighted avg 0.97 0.97 0.97 870


Incorrect identifications during validation

The 25 images in the table below were incorrectly identified during validation.

file actual pred score test_img prediction_exemplar_img
bactrocera_dorsalis_Dacus dorsalis_wing.png bactrocera_dorsalis bactrocera_tryoni 0.5121685
bactrocera_facialis04254492.png bactrocera_facialis bactrocera_kirki 0.7213987
bactrocera_facialis_Bactrocera facialis_wing.png bactrocera_facialis bactrocera_passiflorae 0.9962202
bactrocera_melanotus_Dacus melanotus_wing.png bactrocera_melanotus bactrocera_facialis 0.3934056
bactrocera_passiflorae04204131.png bactrocera_passiflorae bactrocera_tryoni 0.8161860
bactrocera_passiflorae04254454.png bactrocera_passiflorae bactrocera_tryoni 0.5324097
bactrocera_psidii_Bactrocera psidii_wing.png bactrocera_psidii bactrocera_frauenfeldi 0.5140736
trupanea04254299.png trupanea austrotephritis 0.9999244
trupanea04254411.png trupanea austrotephritis 0.9998648
bactrocera_facialis04254491.png bactrocera_facialis bactrocera_passiflorae 0.8368283
bactrocera_facialis04254494.png bactrocera_facialis bactrocera_dorsalis 0.5409914
bactrocera_facialis_Bactrocera facialis_wing.png bactrocera_facialis bactrocera_dorsalis 0.5713534
bactrocera_kirki04257143.png bactrocera_kirki bactrocera_xanthodes 0.5003277
bactrocera_melanotus04254483.png bactrocera_melanotus bactrocera_xanthodes 0.8524919
bactrocera_passiflorae_Bactrocera passiflorae_wing.png bactrocera_passiflorae bactrocera_tryoni 0.9416133
bactrocera_psidii_Bactrocera psidii_wing.png bactrocera_psidii bactrocera_xanthodes 0.4697781
bactrocera_tryoni04204129.png bactrocera_tryoni bactrocera_passiflorae 0.5359533
bactrocera_tryoni04248825.png bactrocera_tryoni bactrocera_dorsalis 0.5752694
bactrocera_facialis_Bactrocera facialis_wing.png bactrocera_facialis bactrocera_passiflorae 0.4918163
bactrocera_facialis_Bactrocera facialis_wing.png bactrocera_facialis bactrocera_passiflorae 0.5807632
bactrocera_facialis04254498.png bactrocera_facialis bactrocera_xanthodes 0.6963255
bactrocera_kirki04254472.png bactrocera_kirki bactrocera_melanotus 0.3627877
bactrocera_passiflorae04204141.png bactrocera_passiflorae bactrocera_facialis 0.5044423
bactrocera_passiflorae_Bactrocera passiflorae_wing.png bactrocera_passiflorae bactrocera_facialis 0.5432149
bactrocera_xanthodes04254470.png bactrocera_xanthodes bactrocera_passiflorae 0.5603202


Representative activation maps for each class

Value for each species is the training accuracy. I am uncertain how to interpret the activation maps. While the wing itself appears to be the most salient object in each image, attention to the finer details within each wing may be needed for better model performance. Is this a result of the small dataset?